Optimization-Driven Hierarchical Learning Framework for Wireless Powered Backscatter-Aided Relay Communications

In this paper, we employ multiple wireless-powered relays to assist information transmission from a multi-antenna access point single-antenna receiver. The wireless can operate in either the passive mode via backscatter communications or active RF communications, depending on their channel conditions and energy states. We aim maximize overall throughput by jointly optimizing transmit beamforming relays’ radio modes operating parameters. Due non-convex combinatorial problem structure, develop novel optimization-driven hierarchical deep deterministic policy gradient (H-DDPG) approach adapt relay strategies. H-DDPG algorithm firstly decomposes binary selection into outer-loop $Q$ -network (DQN) then optimizes continuous relaying strategies using inner-loop DDPG algorithm. Secondly, improve learning efficiency, integrate model-based optimization framework providing better-informed target estimation for DNN training. Simulation results reveal that these two special designs ensure more stable performance achieve higher reward, up 20%, compared conventional model-free approach.